Cross-validation failure: Small sample sizes lead to large error bars
نویسندگان
چکیده
منابع مشابه
Cross-validation failure: Small sample sizes lead to large error bars.
Predictive models ground many state-of-the-art developments in statistical brain image analysis: decoding, MVPA, searchlight, or extraction of biomarkers. The principled approach to establish their validity and usefulness is cross-validation, testing prediction on unseen data. Here, I would like to raise awareness on error bars of cross-validation, which are often underestimated. Simple experim...
متن کاملCross-validation and bootstrapping are unreliable in small sample classification
0167-8655/$ see front matter 2008 Elsevier B.V. A doi:10.1016/j.patrec.2008.06.018 * Corresponding authors. Address: Department o University, Academic Hospital, SE-751 85 Uppasala, Sw fax: +46 18 611 37 03. E-mail addresses: [email protected] (A angstrom.uu.se (M.G. Gustafsson). The interest in statistical classification for critical applications such as diagnoses of patient samples ...
متن کاملIs cross-validation valid for small-sample microarray classification?
MOTIVATION Microarray classification typically possesses two striking attributes: (1) classifier design and error estimation are based on remarkably small samples and (2) cross-validation error estimation is employed in the majority of the papers. Thus, it is necessary to have a quantifiable understanding of the behavior of cross-validation in the context of very small samples. RESULTS An ext...
متن کاملExploratory Factor Analysis With Small Sample Sizes.
Exploratory factor analysis (EFA) is generally regarded as a technique for large sample sizes (N), with N = 50 as a reasonable absolute minimum. This study offers a comprehensive overview of the conditions in which EFA can yield good quality results for N below 50. Simulations were carried out to estimate the minimum required N for different levels of loadings (λ), number of factors (f), and nu...
متن کاملNonparametric Independence Testing for Small Sample Sizes
This paper deals with the problem of nonparametric independence testing, a fundamental decisiontheoretic problem that asks if two arbitrary (possibly multivariate) random variables X,Y are independent or not, a question that comes up in many fields like causality and neuroscience. While quantities like correlation of X,Y only test for (univariate) linear independence, natural alternatives like ...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: NeuroImage
سال: 2018
ISSN: 1053-8119
DOI: 10.1016/j.neuroimage.2017.06.061